Statistical speech translation system based on voice recognition optimization using multimodal sources of knowledge and characteristics vectors

Synergic combination of different sources of knowledge is a key issue for the development of modern statistical translators. Reconnaissance, and thus the translation, can be improved by adding new heuristic characteristics. In this work, a speech translation statistical system that adds additional other-than-voice information in a voice translation system is presented. The additional information serves as a base for the loglinear combination of several statistical models. We obtain the characteristics vectors using a statistical model that is based on the N-best reconnaissance list. We describe the theoretical framework of the problem, summarize the overall architecture of the system, and show how the system is enhanced with the additional information. Our real prototype implements a real-time speech translation system from Spanish to English that is adapted to specific teaching-related environments. Finally, we will provide and explain the system performance results. A tool like the one presented in this article may increase the participation rate of the foreign students to the lecture classes and talks. Keywordsspeech recognition; speech translation; adaptation; pedagogical tool.